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In the present article an attempt has been made to predict the occurrences of 
customers leaving or ‘churning’ a business enterprise and explain the 
possible causes for the customer churning. Three different algorithms are 
used to predict churn, viz. decision tree, support vector machine and rough 
set theory. While two are rule-based learning methods which lead to more 


interpretable results that might help the marketing division to retain or 
hasten cross-sell of customers, one of them is a kernel-based classification 
Keywords: that separates the customers on a feature hyperplane. The nature of 
predictions and rules obtained from them are able to provide a choice 
between a more focused or more extensive program the company may wish 
to implement as part of its customer retention program. 
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1. INTRODUCTION 

Customer relationship management is acknowledged as a popular and effective strategy for creating, 
controlling, and reinforcing strong long-term relationship between the business and the customer. The subject 
has been actively researched and extensively applied to different sectors such as finance, retail businesses or 
telecommunication. One of the main goals of effectively managing the relationship with the customer is 
customer retention. It is a well-established fact that the costs associated with securing a new customer is 
disproportionately more than that of holding back pre-existing customers [1], [2]. It is imperative to explore 
and research up-to-date and robust methods to develop suitable models that could help avoid customer churn. 
In a rapidly growing and transforming market environment, the act of a customer’s changing services has 
been generally observed to be associated with concomitant indicators like one’s transaction history, customer 
service calls, or interactions on the social media, among others [3]. 
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Satisfaction of customers, customer care complains, launch of new products, or upcoming security 
or privacy regulations also affect the movement of customers from one seller to another. The objective of a 
machine learning model to combat churn is identifying early signals of customer’s behavior that could 
probably lead one to snap a continuing business relation. Customer churn or ‘attrition’ seriously affects the 
business of a company, which is why companies strive to precisely identify those customers who are more 
prone to churn during the next month or quarter. There are numerous business models that aim to speculate 
the lifetime value of a customer from the corresponding lifetime period. These models were developed based 
on some criteria such as classifying circular sustainable or specific product lifetime extension (PLE) business 
models, or tapping inevitably into some PLE forms [4]. However, the factor that is filled with the highest 
level of uncertainty in these calculations is the probability of a customer to churn within a certain period. 
Depending through an accurate prediction of the churn probability of customers during a time window, 
companies are likely to gain foot by adopting the right strategy for customer retention programs by offering 
promotions or discounts, through upsell of stock items, or via cross-sell of most-likely-buying items in a 
single basket. 

This article presents the comparative study of three algorithms (decision tree, support vector 
machine and rough set theory) in the prediction of churn events of customers in a telecom dataset and tries to 
find the rules that can generalize the churning behavior of customers. Results have been compared with one 
another, and the advantages of using either of the algorithms in prediction or understanding customer 
behavior to decide future marketing strategies are discussed. The study observed that decision tree (DT) and 
rough set theory (RST) record higher prediction accuracy, while support vector machine (SVM) perform 
better with respect to area under the receiver operating characteristic (ROC) curve. 


2. RELATED WORK 

Several research outcomes have been reported in literature on managing customer relationships and 
the churning pattern of customers. For example, Ascarza et al. [5] reviews the theoretical aspects of the 
problem while Santharam and Krishnan [6] reviewed the prediction techniques applied by researchers in the 
area of customer churn. Vafeiadis et al. [1] described a study on some popular machine learning (ML) 
methods as an application to predict customer churn. The comparative study is based on the dataset from the 
telecommunication industry. A series of Monte Carlo simulations for each model and a wide range of 
parameters are used which demonstrate the supremacy of the boosted versions against other versions. 
SVM-POLY using AdaBoost is one of the best classifiers resulting in almost 97% accuracy and over 84% 
F-measure. Comparative studies among five mostly used classification methods on customer churn indicate 
that their boosting versions have more predictive accuracy. 

The paper authored by Yu et al. [7] worked with an algorithm known as extended support vector 
machine (ESVM) to forecast customer churn prediction. They investigated customers of e-commerce, one of 
the most unstable service sectors. The authors have proposed that ESVM is the best fit to work with 
imbalanced and non-linear nature of churn behavior. According to the paper working with four popularly 
used algorithms, ESVM shows superior results from different accuracy measures such as the rate of hits, 
coverage, or lift. 

To retain old customers than to gain new ones is becoming the new vision of today’s business 
world. From this context, this Xie et al. [8] tried to solve one of the challenging problems of churn 
prediction-imbalanced data distribution. They worked on the churn dataset of Chinese bank customers, 
applying an algorithm improved balanced random forests (IRBF) by combining balanced random forests and 
weighted random forests through assigning a larger weight to the minority class. To get better prediction, the 
paper proposed the addition of integrating methods for sampling and learning processes dependent on cost 
with standard random forests (SRF). They also reported that IRBF produces more accuracy than other 
algorithms like artificial neural network, decision trees, despite its superior results in other fields [9]—[21]. 

Neslin et al. [22] explained how methodology affects the accuracy of predictive model for customer 
churn. The paper analyzed a churn modeling tournament between researchers and practitioners. Both of them 
have to prepare a model for predicting churning customers on two validation databases. Different types of 
calculating approaches confirm that method is an important factor and staying power of the said model is 
another bold line which can be at least 3 months from calibration to compilation of a database. The authors 
have observed a notable difference in the approaches, indicating there is more to prediction than just the 
estimation techniques. 

Burez and Poel [23] illustrated in their paper how to reduce the effect of class imbalance to get more 
real and accurate churn predictions. They have worked with both type of sampling-random and advanced 
sampling, using area under the curve (AUC) and lift evaluation metrics. To find out better results they 
compared two specific algorithms such as gradient boosting machine (GBM) and weighted random forests 
(WRF) (i.e., the gradient boosted model and random forests implementing differential weights) along with 
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some other well-known algorithms. The authors concluded that there is no effective improvement using 
advanced sampling but advised for WRF algorithm in order to get better results. 

The paper of Ahn et al. [24] studied about determinants of churn. They worked on the dataset from 
telecommunication service of Korean market. In order to identify the partial or total defection, the authors 
have recognized a term customer’s status. Uses of the service (frequent or medium or no-use) determine the 
status of customers. Study proposed that the relation between churn determinants and possibility of churn 
calculated through customers’ status change. 

Chen et al. [4] tried to explain customer churn prediction from another angle of reducing noise in 
data range. Databases of business houses are fully occupied with invariant customer data and only specific 
behavior-related information. In this paper the authors approach towards direct use of behavior related 
information, instead of transforming it into an invariant form to get more accuracy. For this innovation they 
developed the novel variant of the SVM with several kernels arranged in different hierarchies, for a three 
phases application. The paper suggests that it gets better prediction of churn following this new method. 


3. MATERIALS AND METHODS 

In the present sections a description of the dataset used in the study is provided, and the methods 
adopted in building the customer churn model that would help to: i) identify certain customers with 
maximum likelihood of churning in a particular month, ii) choose certain features that are more useful in 
predicting the churn behavior, iii) map the relationship between the relevant features and churn probability, 
and iv) draft suitable strategies to combat impending churn and retain valuable customers. The machine 
learning models are introduced in this section. These models will be applied to the dataset, and ensuing 
results obtained from the analysis are presented in the following section. 


3.1. The dataset 

The data has been downloaded from Kaggle website BlastChar [25], and is one of the most popular 
datasets on customer churn. It contains particulars about 7043 customers, and depicts 20 different features of 
a customer, which fall under the following broad categories: demographic information about customers, 
customers’ account-related information, Services they have signed up for, and the class variable Churn. 
Other than these, there is a CustomerID, which identifies each customer with a unique code. We ignore this 
in our analysis. The exhaustive list of features is given in Table 1, along with a short description and the 
categoric values or variable types. 


3.2. Data cleaning and preprocessing 

On exploring the dataset, a few rows with missing values were found in the TotalCharges column. 
One-level deeper introspection revealed that all these customers had a tenure of O months, which means their 
total charges has not been calculated yet and should be 0. Consequently, all these values were coerced to ‘0’. 
Next a check was performed to ensure that all ‘No’ values in the PhoneService feature corresponded to 
‘No phone service’ in the MultipleLines feature. Similarly, it was also checked that all InternetService=‘No’ 
implied OnlineSecurity, OnlineBackup, DeviceProtection, TechSupport, StreamingTV, and treamingMovies 
features had a value of ‘No internet service’ (see features 7, 8 and 9 — 15 in Table 1). A training subset was 
constituted by taking 80% of the customers randomly from the original dataset, while the other 20% served 
as the test set. The efficacy of the models built from the training set is evaluated by predicting churn-status of 
customers in the test dataset. Methodologies outlined in the following subsections are used to build the 
predictive and introspective models from the data. 


3.3. Decision tree 

Decision tree (DT) is a rather pragmatic approach of extracting rules from a dataset Safavian and 
Landgrebe [26]. The rules in fact generalize examples or observations (here ‘customers’) with known class 
values (occurrence or non-occurrence of ‘Churn’). The most popular and simple procedure for constructing a 
decision tree is achieved by partitioning the set of observations along certain conditions or ‘check’s. 

For example, if the set of all 7,043 customers in our dataset are split along the condition 
SeniorCitizen =1, we obtain two branches, with 1,142 customers who are senior citizens, and 5,901 
customers who are not. This can continue recursively, with each branch being further sub-branched along a 
different condition, till a certain “stopping criteria’ is reached. An ideal stopping criterion could be when all 
customers in each terminal branch of the tree belongs to the same class. In this process, every node (starting 
from the root node) becomes a decision node with a specific condition, and every leaf can be identified with a 
specific class label (viz. Churn or Not-Churn). This greedy approach for constructing a tree maps a given set 
of examples to a particular tree, which ultimately classify each customer as having (or not having) churned. 
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Table 1. Variables in the dataset 


No. Variable Description (Values/types) 

1. Customer ID ID of customers (alpha-numeric) 

2. gender Sex (Male, Female) 

3. Senior Citizen Age group of customers (1, 0) 

4. Partner If subscriber lives with partner (Yes, No) 

5. Dependents If he/she has family dependents (Yes, No) 

6. tenure Count of months he/she is a customer (integer) 

7. Phone Service Whether the customer uses a phone (Yes, No) 

8. Multiple Lines If many lines are used (Yes, No, No i phone i service) 
9. Internet Service Type of Internet connection (DSL, Fiber i optic, No) 
10. Online Security Has online security (Yes, No, No i internet i service) 
11. Online Backup Has online backup (Yes, No, No i internet i service) 

12. Device Protection Is the device protected (Yes, No, No i internet i service) 
13. Tech Support Technical support used (Yes, No, No i internet i service) 


14. Streaming TV Subscribed for TV (Yes, No, No i internet i service) 

15. Streaming Movies Subscribed for movies (Yes, No, No i internet i service) 

16. Contract Period of contract (Monthly, Yearly, Biyearly) 

17. Paperless Billing Whether bills avoid paper print (Yes, No) 

18. Payment Method Mode of paying (Electronic i check, mailed i check, Bank i transfer i automatic, Credit i card i automatic) 
19. Monthly Charges Amount billed every month (floating point) 

20. Total Charges Total amount billed (floating point) 

21. Churn Has the customer left the company (Yes, No) 


The method is termed as ‘greedy’ from the consideration that at every step the objective is to 
maximize separation between the classes. In other words, among all possible conditions available for 
branching, the one that achieves maximum class separation is chosen. This is done with the help of a suitable 
metric such as Gini Impurity or Entropy that purportedly measures the impurity inherent in a node, or lack of 
‘class separation’ as (1) and (2): 


Ginilmpurity : Ie (p) = Lies pi — Di) (1) 
Entropy : H(p) = — Viz pilog2p; (2) 


where p is the probability of finding an observation of class i at any particular node or leaf, and c is the 
cardinality of the set of classes in the dataset. In the present scenario, c=2 with just two classes (Churn and 
Not Churn). 


3.4. Support vector machine 

Support vector machine (SVM) is a mathematically oriented procedure to divide the space of objects 
belonging to different classes with suitable lines, planes, or hyperplanes. The procedure was introduced by 
Vapnik and Chervonenkis [27] as a method of statistical learning in the 1960’s. Finding the hyperplane w 
essentially involves solving the optimization problem. 


min(w-w) s.t. y'((w-x')+b)>1 (3) 


w,b 


Where x! and y! are the input and target values of the ith object, with the index of objects spanning the range 
i = 1,2,..,N. 

Implementation of the SVM classifier was achieved with the help of the SVC module of the 
scikitlearn Python library Nelli [28]. A polynomial kernel was used with a degree varying from 1 to 5. The 
results are presented in the results and discussion section. 


3.5. Rough set theory 

Rough set theory (RST) is a recent paradigm that is gaining popularity in data science and machine 
learning. It was proposed by Pawlak [29]. An object or observation is considered to belong to some concept 
or class, depending on incomplete information about the object. The information system J is expressed as a 
crisscross of objects and input + target features (or attributes). 


J={U, AU {d} (4) 


Where U is the set of all observations, A is the aggregate of input features, and {d} is the target variable. 
Rules may be inferred from a given set of data using features from the reduct R C A which contains 
sufficient information to derive the class of objects with the same accuracy as A. The procedure of obtaining 
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rules is preceded by discretizing the continuous features into suitable intervals, and subsequently selecting a 
bare minimum set of features that classifies the dataset with the same accuracy as the entire set of attributes. 
Several algorithms exist in the R RoughSets library Riza et al. [30] to discretize features and select them into 
a reduct. The global discernibility heuristic and the quick reduct algorithms Shen and Chouchoulas [31] were 
used in the present analysis respectively for discretization and feature selection. Both are supervised methods, 
categorizing and selecting features keeping in view the final task of classification. For extracting rules, the 
AQRules method Michalski et al. [32] was used with the confidence threshold set to 0.8 and times covered as 
1, to get only the most accurate rules with no oversampling. 


3.6. Accuracy metrics 
Certain metrics are customarily used to evaluate the performance of a prediction or classification 
model, particularly following a specific algorithm. They are discussed in the following paragraphs. 

— Confusion matrix: It gives the overall picture of the correct and wrong predictions of each class. For a 
binary class problem, a confusion matrix shows the number of true negatives (TN), false negatives (FN), 
true positives (TP) and false positives (FP). The typical form of a confusion matrix for the case where 
there are just two classes, 0 and 1, is shown in Table 2. 


Table 2. Confusion Matrix for two classes 


Predicted Class 
Actual Class 0 1 
0 TN FP 
1 FEN TP 


— Accuracy: It is the simplest metric for assessing the correctness of predictions across all classes. For a 
binary class problem, it takes the expression: 


TP+TN 


Accuracy = ———__—— 
Y = Tp 4EN+EP+TN 


(5) 


— Specificity: Specificity is the ability of a classifier to reject false prediction of the actual negatives and 
predict all negative results correctly. It is the ratio of negative observations predicted correctly to the total 
number of negative observations in the dataset. 


TN 


Specificity = TNŁFP 


(6) 

— Sensitivity: It measures how well a model senses positive values or the degree of response of an 
algorithm. It calculates the proportion of true positives to the total number of actually positive 
observations. 


(7) 


Pee T 
Sensitivity = TPN 


— Area under the ROC curve (AUC-ROC): It Is the area under the receiver operating characteristic (ROC) 
curve, which is generated by the graph of sensitivity, also called true positive rate (TPR) against the false 
positive rate (FPR), taken at different thresholds. TPR and FPR may be expressed as (8) and (9). 


TruePositiveRate=TP/(TP+FN) (8) 
FalsePositiveRate=FP/(FP+TN) (9) 
The above metrics are used in the following section to integrate the results obtained from the models 
developed, and measure their abilities in predicting churned customers correctly. The set of churned 
customers (Churn = Yes) is taken as the positive class. 
— Accuracy and support of a rule: They are two important parameters that determine the significance of an 


if-then rule of the form: 


If ọ then y (10) 
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The two quantities are defined as (11) and (12): 


[ligila haill 


Accuracyofarule = 
4 Jle] 


(1) 


Supportofarule = | ||ġ|| 9 [lll | (12) 


where ||ọ|| is the set of objects obeying the condition 9, and ||yl|| is the set of objects that fall into the class w, 
and | - | gives the number of elements a set has. Accuracy of any rule denotes the confidence with which the 
tule classifies a set of objects into a specific class, while coverage is the number of objects correctly 
predicted by it. These two qualifiers are provided at the end of each rule presented in the next section. 


4. RESULTS AND DISCUSSION 

We first explore to visualize the percentage of subscribers who have churned in the entire dataset 
with the help of a pie-chart as shown in Figure 1. Subsequently the distribution of tenure and Monthly 
Charges are explored with the help of density plots as shown in Figure 2. The three different machine 
learning algorithms (DT, SVM and RST) elucidated in materials and methods section were trained on 80% of 
the dataset, and the trained models are used to predict the churn status on the remaining 20% of the 
customers. 


% of Churn in dataset 


73.50% 


s Chums = Does not Churn 


Figure 1. Percentage of customers who have [not] churned 


Variation of "tenure" for (Un-]Churned customers Variation of "Monthly Charges" for [(Un-]Churned customers 
-== Chumed -= = Churned 
—— UnChurned 0.01757 UnChurned 


0.0150 
0.0125 
F 0.0100 
= 0.0075 
0.0050 


0.0025 


0.0000 
0 20 40 60 80 -20 0 20 40 60 80 100 120 140 
tenure Monthly Charges 


Figure 2. Distribution of [un] churned customers along the feature’s tenure and monthly charges 


The parameter tuning parameters of the three algorithms are explored across a wide range to get the 
optimal performance as: i) DT: depth of tree = 3 (tried 2, 3, ..., 9); ii) SVM: degree of polynomial = 2 (tried 
1, 2, ..., 5); and iii) RST: confidence threshold = 0.8 (tried 0.7, 0.75, 0.8, 0.85, 0.9, 0.95). 
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The confusion matrix that provides the entire picture of the predicted classes vis-a-vis the actual 
classes of the customers both in the training and test sets are presented in the first two rows of Table 3. The 
confusion matrix that provides the entire picture of the predicted classes vis-a-vis the actual classes of the 
customers both in the training and test sets are presented in the first two rows of Table 3. In the next four 
rows the accuracy, sensitivity, specificity, and area under the ROC curve obtained from the prediction on the 
three algorithms on the training and test sets are also presented. The best accuracy is obtained from the 
decision tree and rough set theory, with nearly 80% accuracy on both the training and validation subsets. 


Table 3. Confusion matrices and prediction accuracies from different machine learning algorithms 


Actual Training Set Test Set 
Class DT SVM RST DT SVM RST 
Prediction: 0 1 0 1 0 1 0 1 0 1 0 1 
0 3884 255 2760 1379 4096 51 971 64 690 345 993 34 
1 930 565 431 1064 1081 406 222- "152 88 286 287 95 
Accuracy 79.0% 67.9% 79.9% 79.7% 69.3% 71.2% 
Sensitivity 93.8% 66.7% 98.8% 93.8% 66.7% 96.7% 
Specificity 37.8% 71.2% 27.3% 40.6% 76.5% 24.9% 
AUC-ROC 65.8% 68.9% 63.0% 67.2% 71.6% 60.8% 


However, in respect of AUC-ROC, support vector machine shows a slightly enhanced performance 
compared to the other two approaches by around 3 to 4%. The specificity and sensitivity of SVM is also 
more stabled than DT or RST which attain more than 90% sensitivities at the cost of specificities less than 
40%. This tendency is the most marked in RST where specificity falls below 30% to get a 97% accuracy. 

This makes SVM predictions suitable when a more extensive retention program is in place, as it is 
able to identify 286 out of 374 churning customers in the test set, at the same time wrongly bringing in 345 
loyal customers under this umbrella. On the other hand, DT or RST prediction can be more useful when the 
company would not afford to waste its resources for a retention program on its loyal customers rather wishes 
to take a more focused churn-intensive retention program, not only because fewer (around 100 to 150) churns 
are identified here, but also since these two algorithms bring in much fewer misclassified (35 to 65) 
customers under this program. 

No signs of overfitting are apparent in the results, as the test set metrics are very close to the training 
set metrics. However, in the different parametric tuning, it was observed that a decision tree of depth more 
than 7 or a kernel polynomial of degree more than 4 seems to set in overfitting to some extent. Finally, the 
most significant rules obtained from DT and RST for the Churn class is presented below. The figures in 
square brackets at the end of each rule denote the rule accuracy and coverage explained in (12). The rules 
generated through the Decision Tree model (of depths 3, 4 and 5) are as follows: 


If Contract = Monthly and Internet Service = Fiber optic and tenure < 
15.5 then Churn [69%, 565]. 

If Contract = Monthly and Internet Service = Fiber optic and tenure < 
15.5 and TotalCharges < 120 then Churn [86%, 161]. 

If Contract = Monthly and Internet Service = Fiber optic and tenure < 
15.5 and TotalCharges > 120 and MonthlyCharges > 
97.50 then Churn [83%, 69]. 

If Contract = Monthly and Internet Service + Fiber optic and tenure < 
4.5 and MonthlyCharges < 55.20 then Churn [57%, 156]. 


The rules obtained from Rough Set analysis are as: 


If gender = Female and TotalCharges E (309, 1180] and MonthlyCharges E 
(90.2, 100) then Churn [88%, 38]. 
If tenure < 3 and StreamingTV = Yes and SeniorCitizen = 1 then Churn [88%, 28]. 
If StreamingTV = Yes and TechSupport = No and tenure € 
(3,6.5) and StreamingMovies = Yes then Churn [84%, 31]. 
If PaymentMethod = Electronic check and OnlineBackup = No and Monthly Charges E€ 
(90.2, 100) and tenure < 3 then Churn [83%, 29]. 
If InternetService = Fiber optic and tenure € (3,6.5) and gender = 
Female and StreamingMovies = Yes then Churn [88%, 28]. 
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If gender = Female and MonthlyCharges € (73.6,90.2) and TotalCharges < 
91.3 and StreamingMovies = No then Churn [90%, 28]. 

If tenure € (3,6.5) and MultipleLines = Yes and gender = Male and OnlineSecurity = 
No then Churn [81%, 25]. 


The rules from DT obtained are broader in scope, involve few features, but less accurate. On the 
other hand, the rules from RST are more specific, generally more accurate, however supported by fewer 
objects. Though this can be tuned by the tuning parameters of the two algorithms, the contrast and 
complimentary is intentionally maintained with specific business strategies in mind. This can be viewed as a 
typical advantage of pluralistic analysis with several different algorithms set to appropriate parameters, where 
results of different nature are obtained, and the company executive gets the choice of selecting the right kind 
of rules that would best fit its present purpose. For example, if the management decides to reach as many 
customers as possible to reduce churn (with an extensive program) rules from DT might appear more 
appropriate as each of the four rules presented above would help reach almost a thousand customers, but 25% 
of its resources on customer retention might go wasted, since the accuracy of the rules are low. However, if 
on some other occasion the management is obliged to reach only a few customers (maybe as a pilot study to 
see how they respond to discounts and offers or similar retention programs) the marketing executive is free to 
device their benefits targeting only a few chosen sections of the churning customers, but where the 
probability of churn is much higher, and only 10 to 15% of the resources may be wasted. 


5. CONCLUSION 

The problem of customers leaving a particular service and migrating to its competitors involve a 
challenging aspect of customer relationship management. Churn in customers can be prevented if a proper 
strategy is implemented to detect early chances of churn, followed by suitable retention programs. The paper 
compares the performance of three different algorithms on their ability to predict customer churn from a 
publicly available dataset. The respective parameters of each algorithm are properly tuned to achieve optimal 
prediction. It is observed that DT and RST record higher prediction accuracy, while SVM perform better with 
respect to area under the ROC curve. The specificity and sensitivity of SVM are more balanced, whereas DT 
and RST are able to score better sensitivities at the cost of lower specificity. Rules obtained from DT and RST 
are enlisted with accompanying accuracy and coverage values. The different nature of rules revealed by the 
two algorithms gives the management a chance to choose between extensive and intensive programs on 
different sections of its customers at appropriate implementation levels of customer retention strategies it 
might be interested in adopting. 
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